Move force merge from the downsmapling request to the ILM action #135834

gmarouli · 2025-10-02T09:16:12Z

In this PR we move the force-merge operation from the downsampling request to the ILM action.

Our goal is to decouple the downsampling operation from the force-merge operation. With this change the downsampling request is responsible to ensure that the downsampled index is refreshed and flushed but not to force merge it.

We believe that most of the time this is not necessary, and executing the force-merge operation unnecessarily can increase the load on the cluster.

To preserve backwards compatibility we move the responsibility to execute the existing force merge to the downsample ILM action and we make it configurable. By default, it will run but a user can disable it just as they can with a searchable snapshot.

"downsample": {
  "fixed_interval": "1h",
  "force_merge_index": false
}

Update

As a follow up of this PR, we pose the question is the force merge in the downsample action intentional and useful?

To answer this question, we extend time series telemetry. We define that the force merge step in the downsample ILM action is useful, if this is the only force merge step operation before a searchable snapshot.

Effectively, by this definition, we argue that the force merge in downsampling is not an intentional operation the user has requested but only the result of the implementation. We identify the biggest impact of removing it to be a searchable snapshot, but if the searchable snapshot performs its own force merge (and more performant force merge #133954) then we could skip this operation in the downsample action altogether.

Fixes: #135618

github-actions · 2025-10-02T10:22:22Z

🔍 Preview links for changed docs

docs/reference/elasticsearch/index-lifecycle-actions/ilm-downsample.md

github-actions · 2025-10-02T10:22:23Z

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

Check out the cumulative docs guidelines
Reach out in the #docs Slack channel

gmarouli · 2025-10-02T15:01:44Z

~~As it says in the description, I have one more idea for a test but I thought of it after I requested your review. So, no rush tomorrow the final test should be there too.~~

This idea won't work, because we cannot influence the segments of the downsampled index.

elasticsearchmachine · 2025-10-02T15:11:35Z

Pinging @elastic/es-storage-engine (Team:StorageEngine)

elasticsearchmachine · 2025-10-02T15:11:35Z

Pinging @elastic/es-data-management (Team:Data Management)

nielsbauman

Overall LGTM, just left a few small comments :)

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ilm/DownsampleAction.java

nielsbauman · 2025-10-02T16:34:58Z

...n/downsample/src/main/java/org/elasticsearch/xpack/downsample/TransportDownsampleAction.java

-    private boolean isForceMergeEnabled(Map<String, Object> sourceIndexMappings) {
-        if (sourceIndexMappings.containsKey("_meta")) {
-            if (sourceIndexMappings.get("_meta") instanceof Map<?, ?> metadataMap) {
-                var enabledForceMergeValue = metadataMap.get("downsample.forcemerge.enabled");


Just double-checking: do we have any documentation on this "setting"? It's in a weird place so I doubt we care much about it, but just wanted to double-check that we're ok with not supporting this configuration any more.

This setting is not documented on purpose. We added it to allow users that cannot upgrade yet from 8.19, to test the impact of skipping the force-merge request. It's just a workaround since the proper way, which we implement with this PR, cannot be backported.

...multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/DownsampleActionIT.java

dakrone

LGTM also in general. I have the same question as Niels about not looking at the metadata any more, is that going to be considered a breaking change, since a user that has that configured as false would start having their downsampled indices force merged after this change?

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ilm/DownsampleAction.java

gmarouli · 2025-10-03T06:51:26Z

LGTM also in general. I have the same question as Niels about not looking at the metadata any more, is that going to be considered a breaking change, since a user that has that configured as false would start having their downsampled indices force merged after this change?

This is an undocumented workaround to allow users in 8.19.x to experiment with disabling force merge and see the impact. Considering this, I think this change is reasonable. When they upgrade they can update their policies to skip the force-merge the right "way".

elasticsearchmachine · 2025-10-03T10:39:27Z

Hi @gmarouli, I've created a changelog YAML for you.

gmarouli · 2025-10-08T15:30:53Z

@dakrone and/or @nielsbauman can you take another look?

elasticsearchmachine · 2025-10-08T15:40:16Z

Hi @gmarouli, I've updated the changelog YAML for you.

In this PR, we propose to add practical tips for downsampling. For now this includes, a guideline on how to choose the downsampling interval. And then specifically for ILM, an explanation on how downsampling relates with tiers. After elastic/elasticsearch#135834, we should also add here the option to disable force merge. --------- Co-authored-by: Kostas Krikellas <[email protected]> Co-authored-by: Marci W <[email protected]>

nielsbauman

LGTM. The only recommendation I have left is to run the test classes you changed a few thousand iterations, to ensure there's no combination of random values that results in a test failure.

Apologies for the late review, and thanks a lot for working and iterating on this!

...n/core/src/main/java/org/elasticsearch/xpack/core/action/TimeSeriesUsageTransportAction.java

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ilm/DownsampleAction.java

...n/core/src/main/java/org/elasticsearch/xpack/core/action/TimeSeriesUsageTransportAction.java

nielsbauman

I left one small typo suggestion, other than that LGTM!

...n/core/src/main/java/org/elasticsearch/xpack/core/action/TimeSeriesUsageTransportAction.java

Co-authored-by: Niels Bauman <[email protected]>

gmarouli · 2025-10-14T09:37:11Z

Thank you for the review @nielsbauman , no worries about the delay, there were many things in progress so I wasn't blocked!

gmarouli added 2 commits October 2, 2025 09:39

Add a configurable force merge step to ILM DownsampleAction

4d6e2d2

Remove force merge from downsampling API

a73737b

elasticsearchmachine added the v9.3.0 label Oct 2, 2025

Update docs

6951827

merge with main

e4064bc

gmarouli requested review from dakrone, martijnvg and nielsbauman October 2, 2025 14:55

gmarouli added :Data Management/ILM+SLM Index and Snapshot lifecycle management :StorageEngine/Downsampling Downsampling (replacement for rollups) - Turn fine-grained time-based data into coarser-grained data labels Oct 2, 2025

gmarouli added 2 commits October 2, 2025 17:56

merge with main

aa47a9d

bump transport version

5fcbf47

gmarouli marked this pull request as ready for review October 2, 2025 15:11

elasticsearchmachine added Team:Data Management Meta label for data/management team Team:StorageEngine labels Oct 2, 2025

nielsbauman reviewed Oct 2, 2025

View reviewed changes

dakrone reviewed Oct 2, 2025

View reviewed changes

gmarouli added 4 commits October 3, 2025 09:54

merge with main

3ab9134

Convert forceMerge flag primitive to Boolean

c14a301

Add telemetry for the downsampling force merge

d0d0b0f

Merge branch 'main' into downsampling/move-force-merge-to-ilm

ada4cf7

gmarouli added >enhancement >refactoring and removed >enhancement labels Oct 3, 2025

gmarouli added >enhancement and removed >refactoring labels Oct 3, 2025

Update docs/changelog/135834.yaml

c2d3cdc

gmarouli requested review from dakrone and nielsbauman October 3, 2025 10:59

Update changelog

2b8e110

gmarouli mentioned this pull request Oct 6, 2025

Add practical tips for downsampling. elastic/docs-content#3340

Merged

gmarouli added 6 commits October 6, 2025 17:51

merge with main

5a8419d

merge with main

7f574d2

Add version tag

be659d3

Add force merge tasks to the test

7233dca

Merge branch 'main' into downsampling/move-force-merge-to-ilm

d5c2f26

merge main

c12b23c

gmarouli self-assigned this Oct 8, 2025

Update docs/changelog/135834.yaml

340d125

merge with main

6bdd8b6

nielsbauman approved these changes Oct 10, 2025

View reviewed changes

gmarouli added 2 commits October 14, 2025 11:27

merge with main

f670902

Apply review feedback.

c006161

nielsbauman approved these changes Oct 14, 2025

View reviewed changes

...n/core/src/main/java/org/elasticsearch/xpack/core/action/TimeSeriesUsageTransportAction.java Outdated Show resolved Hide resolved

Apply suggestion

09ec3bb

Co-authored-by: Niels Bauman <[email protected]>

Merge branch 'main' into downsampling/move-force-merge-to-ilm

27177a7

gmarouli added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Oct 14, 2025

elasticsearchmachine merged commit b0ef2b4 into elastic:main Oct 14, 2025
34 checks passed

gmarouli deleted the downsampling/move-force-merge-to-ilm branch October 14, 2025 12:31

Move force merge from the downsmapling request to the ILM action #135834

Move force merge from the downsmapling request to the ILM action #135834

Uh oh!

Conversation

gmarouli commented Oct 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Preview links for changed docs

Uh oh!

github-actions bot commented Oct 2, 2025

ℹ️ Important: Docs version tagging

When to use applies_to tags:

What NOT to do:

🤔 Need help?

Uh oh!

gmarouli commented Oct 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticsearchmachine commented Oct 2, 2025

Uh oh!

elasticsearchmachine commented Oct 2, 2025

Uh oh!

nielsbauman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

nielsbauman Oct 2, 2025

Choose a reason for hiding this comment

Uh oh!

gmarouli Oct 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dakrone left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gmarouli commented Oct 3, 2025

Uh oh!

elasticsearchmachine commented Oct 3, 2025

Uh oh!

gmarouli commented Oct 8, 2025

Uh oh!

elasticsearchmachine commented Oct 8, 2025

Uh oh!

nielsbauman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nielsbauman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gmarouli commented Oct 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gmarouli commented Oct 2, 2025 •

edited

Loading

github-actions bot commented Oct 2, 2025 •

edited

Loading

gmarouli commented Oct 2, 2025 •

edited

Loading

dakrone left a comment •

edited

Loading